Learning Procedural Planning Knowledge in Complex Environments

LEARNING PROCEDURAL PLANNING KNOWLEDGE IN COMPLEX ENVIRONMENTS by Douglas John Pearson Chair: John E. Laird In complex, dynamic environments, an agent's knowledge of the environment (its domain knowledge) will rarely be complete and correct. Existing approaches to learning and correcting domain knowledge have focused on either learning procedural knowledge to directly guide execution (e.g. reinforcement learners) or learning declarative planning knowledge (e.g. theory revision systems). Systems that only learn execution knowledge are generally only applicable to small domains. In these domains it is possible to learn an execution policy that covers the entire state space, making planning unnecessary. Conversely, existing approaches to learning declarative planning knowledge are applicable to large domains, but they are limited to simple agents, where actions produce immediate, deterministic e ects in fully sensed, noise-free environments, and where there are no exogenous events. This research investigates the use of procedural knowledge to support the learning of planning knowledge in large and complex environments. We describe a series of environmental properties that constrain learning and are violated by existing approaches to learning planning knowledge. We then present an operator-based representation for planning knowledge that is su ciently expressive to model complex, conditional actions that produce sequential e ects over time. We then present IMPROV, a system for learning and correcting errors in this planning knowledge that only requires procedural access to the knowledge. This procedural restriction ensures that learning remains tractable, even over these large, expressive representations. We rst explain how IMPROV incrementally learns operator precondition knowledge. We then demonstrate how a hierarchical, operator-based representation can be used to reduce the problem of learning operator e ects to the problem of learning operator preconditions. This result allows IMPROV to use a single learning method to learn both operator preconditions and e ects. This also allows IMPROV to learn complex models of actions that produce conditional or sequential e ects. Finally, we test the system in two sample domains and empirically demonstrate that it satis es many of the constraints faced by learning agents in complex and challenging environments. c Douglas John Pearson 1996 All Rights Reserved To Helen For each and every, perfectly ordinary and perfectly wonderful day.